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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE MONTH{S) FROM 

THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX {6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply w/ithin the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent temi adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to communicationfs) filed on 30 January 2004 . 
2a)KI This action is FINAL. 2b)n This action Is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) EI Claim(s) 1-6.12.13 and 28-91 is/are pending in the application. 

4a) Of the above claim{s) 

5) 0 Claim(s) is/are allowed. 



is/are withdrawn from consideration. 



6)^ Claim(s) 1-6.12.13 and 28-91 is/are rejected. 
?)□ Claim{s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: 8)0 accepted or bjD objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 Certified copies of the priority documents have been received. 

2.n Certified copies of the priority documents have been received in Application No. . 



3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Response to Applicant's Remarks 

1 . Applicant's arguments submitted on 01/30/04 with respect to claims 73-91 have 
been reconsidered but are not deemed persuasive for the reasons set forth below. 

Applicant argues (under REMARKS section) that Gabby does not teach or 
suggest graphically ordering the annotations in a desired order of presentation in 
response to user input; reordering the plurality of identifiers in accordance with user 
input to change the order in which the media segments are to be presented; receiving 
an indication of particular user-selected annotations of a plurality of annotations, 
wherein each of the plurality of annotations corresponds to a media stream or to one or 
more media streams; and seamlessly providing one or more of, the user-selected 
annotations, and at least a portion of the media stream corresponding to each of the 
user-selected annotations. 

Examiner respectfully disagrees. Gabbe teaches: 

"The natural partitioning of a complex entity into episodes, goes hand-in-hand with a type of "associative retrieval", i.e., the idea 
that elements of an episode can be used to trigger recall of the entire or of related episodes [LAMM-92]. Thus, for example, 
remembering -visually - the point at which someone joined a meeting late, can trigger recall of the entire discussion - words - 
that ensued as a result. In the context of multimedia systems, we use the term multimodal retrieval as a catch-all for retrieval 
mechanisms that allow retrieval mechanisms that allow retrieval of one type of media or data by use of another type " ... "As 
shown in Figure 1 , lERS can be associated with every type of element in the recorded system. In the area of control events, for 
example, one can easily imagine using symbolic lERs to represent or user launching /killing applications, typing in special 
annotations, or initiating a phone call. Generating a visual icon based upon one or more frames of a video clip is an example of a 
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visual lER [TEOD-93]." ... "As discussed in the subsequent section on our multimedia-conference-recording application, the 
icons may be presented to the user as a film strip (iconic slider) through which the user can scroll or jump (see Figure 2). Since 
each IHR is associated with a Last_Record_Generated vector, a user can redisplay all or part of the entire state of the recorded 
system associated with with the icon by "selecting" it. The icons also serve as easily identifiable anchor points for attaching 
annotations, control events, and index records to episodes." ... "To form building blocks for episodes, we need to process these 
streams into segments on the fly and augment them with ancillary information. We call the augmented stream segments clips." ... 
"A clip is composed of a video segment augmented by such items as an lER and a scene label.". 

The quoted text excerpts clearly indicates tliat multimedia streams are divided 
into episodes. The episode consists of a plurality of continuous/seamless frames. The 
lER is associated with an episode. The annotations can be typed in an lER. Therefore 
lER is basically a visual annotation to the associated episode. lERs combined with the 
episodes are called clips. An user, who may joins the conference late is presented with 
a table of lERs. The user can scroll or jump through the icons to catch up with the 
conference and play the missed episodes/clips by selecting the icons. Therefore the 
user can change the natural order of playing the episodes/clips. Within each episode 
the playback is seamless. The lERs and their associated episodes must be stored in a 
persistent storage and othenA^ise they would be lost. The file/ databases are universally 
age-old mechanism to store information. The applicant does not claim any special 
features of a database that enhanced the multimedia retrieval process. The applicant 
merely uses database to store the multimedia information. The lERs are displayed in 
some kind of window, allowed by window system (Figure 2) (e.g. Microsoft windows 
programming, Motif GUI standards in UNIX or Java GUI standard). Dragging is allowed 
by all of the above GUI standards. Therefore a user can drag the icons to reorient them 
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along the desired sequence and replay the interesting portion/clips of the conference in 
the desired order. User also can select particular icons by using a mouse. Dragging 
visual entities in a window is a very basic feature of a GUI window. The icons with 
attached annotation serve as easily identifiable anchor points control events and index 
records to episodes. Therefore these visual annotations with scene label work as 
identifiers to the seamless frames of the episodes. 



Double Patenting 

2. The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or Improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. See In re Goodman, 1 1 
F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 
USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 
1982); In re Vogeh 422 F.2d 438, 164 USPQ 619 (CCPA 1970);and. In re Thorington, 
418 F.2d 528. 163 USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .321(c) may be 
used to overcome an actual or provisional rejection based on a nonstatutory double 
patenting ground provided the conflicting application or patent is shown to be commonly 
owned with this application. See 37 CFR 1 .130(b). 

Effective January 1, 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 



3. Claims 1-6,12-13 and 28-91 are rejected under the judicially created doctrine of 
obviousness-type double patenting as being unpatentable over claims 1-18 of 
Purnaveja et al. (U.S. Patent No. 6.006,241 and Purnaveja hereinafter), claims 1-18 of 
Purnaveja et al. (U.S. Patent No. 6,230,172 and Purnaveja hereinafter), claims 1-39 of 
Chaddha et al. (U.S. Patent No. 6,173,317 and Chaddha hereinafter), claims 1-25 of 
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Gupta et al. (U.S. Patent No. 6,484,156 and Gupta hereinafter). Although the 
conflicting claims are not identical, they are not patentably distinct from each other 
because the subject matter claimed in the instant application is fully disclosed in the 
patent and is covered by the patent since the patent and the instant application are 
claiming common subject matter and they are substantially similar in scope and they 
use the same limitations, using varying terminology. 

Claims 1-6,12-13 and 28-91 of the instant application corresponds to claims 1- 
18 of the 6,006,241 patent, claims 1-18 of 6,230,172 patent, claims 1-39 of 6,173,317 
patent and claims 1-25 of 6,484,156 patent. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

5. Claims 1-6,12-13 and 28-91 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Gabbe et al. ("Towards Intelligent Recognition of Multimedia Episodes in 
Real-Time Applications" 1994 ACM 0-89791-686-7/94/0010 and Gabbe hereinafter). 

As to claims 1-6.12-13 and 28-91, Gabbe teaches receiving an indication of a 
plurality of annotations selected by a user, wherein each of the plurality of annotations 
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corresponds to a media stream or to one or more media streams (i.e. the movie parser 
segments the video material into chunks of contextual information and also the system 
provides video recording, indexing, annotating, browsing tools and playback features; In 
this system the user may select a video image and annotate the image with text for 
subsequent use (i.e. the system allows the user to parse the seamless media stream 
into images and allows the user to index and annotate the images and also allows the 
user to playback the movie/seamless media stream with textual annotations)) (page 
235, col 1-2); presenting a plurality of annotation identifiers (i.e. indexes) to the user; 
allowing the ordering of the plurality of annotation identifiers to be changed by the user 
(i.e. the system allows the user create indexes and hence change the ordering of 
images and annotations) (page 235, col 1-2); seamlessly providing one or more of , the 
plurality of annotations, and at least a portion of the media stream corresponding to 
each of the plurality of annotations; wherein seamlessly providing comprises seamlessly 
providing the one or more of the plurality of annotations and the portions of the media 
stream corresponding to each of the plurality of annotations in an order defined by the 
order of the plurality of annotation identifiers (page 235, col 1-2). 



Conclusion 

6. THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded 
of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
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TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Apu M. Mofiz whose telephone number is (703) 605- 
4240. The examiner can normally be reached on Monday - Thursday 8:00 A.M. to 4:30 
P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dov Popovici can be reached at (703) 3053830. The fax numbers for the 
group is (703) 746-7239. 

Any inquiry of a general nature or relating to the status of this application should 
be directed to the Group receptionist whose telephone number is (703) 305-9600. 
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